18 research outputs found
ResolvNet: A Graph Convolutional Network with multi-scale Consistency
It is by now a well known fact in the graph learning community that the
presence of bottlenecks severely limits the ability of graph neural networks to
propagate information over long distances. What so far has not been appreciated
is that, counter-intuitively, also the presence of strongly connected
sub-graphs may severely restrict information flow in common architectures.
Motivated by this observation, we introduce the concept of multi-scale
consistency. At the node level this concept refers to the retention of a
connected propagation graph even if connectivity varies over a given graph. At
the graph-level, multi-scale consistency refers to the fact that distinct
graphs describing the same object at different resolutions should be assigned
similar feature vectors. As we show, both properties are not satisfied by
poular graph neural network architectures. To remedy these shortcomings, we
introduce ResolvNet, a flexible graph neural network based on the mathematical
concept of resolvents. We rigorously establish its multi-scale consistency
theoretically and verify it in extensive experiments on real world data: Here
networks based on this ResolvNet architecture prove expressive; out-performing
baselines significantly on many tasks; in- and outside the multi-scale setting
Geometrically Consistent Partial Shape Matching
Finding correspondences between 3D shapes is a crucial problem in computer
vision and graphics, which is for example relevant for tasks like shape
interpolation, pose transfer, or texture transfer. An often neglected but
essential property of matchings is geometric consistency, which means that
neighboring triangles in one shape are consistently matched to neighboring
triangles in the other shape. Moreover, while in practice one often has only
access to partial observations of a 3D shape (e.g. due to occlusion, or
scanning artifacts), there do not exist any methods that directly address
geometrically consistent partial shape matching. In this work we fill this gap
by proposing to integrate state-of-the-art deep shape features into a novel
integer linear programming partial shape matching formulation. Our optimization
yields a globally optimal solution on low resolution shapes, which we then
refine using a coarse-to-fine scheme. We show that our method can find more
reliable results on partial shapes in comparison to existing geometrically
consistent algorithms (for which one first has to fill missing parts with a
dummy geometry). Moreover, our matchings are substantially smoother than
learning-based state-of-the-art shape matching methods
SIGMA: Scale-Invariant Global Sparse Shape Matching
We propose a novel mixed-integer programming (MIP) formulation for generating
precise sparse correspondences for highly non-rigid shapes. To this end, we
introduce a projected Laplace-Beltrami operator (PLBO) which combines intrinsic
and extrinsic geometric information to measure the deformation quality induced
by predicted correspondences. We integrate the PLBO, together with an
orientation-aware regulariser, into a novel MIP formulation that can be solved
to global optimality for many practical problems. In contrast to previous
methods, our approach is provably invariant to rigid transformations and global
scaling, initialisation-free, has optimality guarantees, and scales to high
resolution meshes with (empirically observed) linear time. We show
state-of-the-art results for sparse non-rigid matching on several challenging
3D datasets, including data with inconsistent meshing, as well as applications
in mesh-to-point-cloud matching.Comment: 14 page
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go
We present NeuroMorph, a new neural network architecture that takes as input
two 3D shapes and produces in one go, i.e. in a single feed forward pass, a
smooth interpolation and point-to-point correspondences between them. The
interpolation, expressed as a deformation field, changes the pose of the source
shape to resemble the target, but leaves the object identity unchanged.
NeuroMorph uses an elegant architecture combining graph convolutions with
global feature pooling to extract local features. During training, the model is
incentivized to create realistic deformations by approximating geodesics on the
underlying shape space manifold. This strong geometric prior allows to train
our model end-to-end and in a fully unsupervised manner without requiring any
manual correspondence annotations. NeuroMorph works well for a large variety of
input shapes, including non-isometric pairs from different object categories.
It obtains state-of-the-art results for both shape correspondence and
interpolation tasks, matching or surpassing the performance of recent
unsupervised and supervised methods on multiple benchmarks.Comment: Published at the IEEE/CVF Conference on Computer Vision and Pattern
Recognition 202